Edge-preserving down/upsampling for depth map compression in high- efficiency video coding

An efficient down/upsampling method to compress a depth map efficiently within the high-efficiency video coding (HEVC) framework is presented. A different edge-preserving depth upsampling method is proposed by using both the texture and depth information. We take into account the edge similarity between depth maps and their corresponding texture images as well as the structural similarity among depth maps to build a weight model. Based on the weight model, the optimal minimum mean square error upsampling coefficients are estimated from the local covari-ance coefficients of the downsampled depth map. The upsampling filter is combined with HEVC to increase coding efficiency. The objective results demonstrate that we achieve a maximum bit rate saving of 32.2% compared to full resolution method and 27.6% compared to a competing depth down/upsampling method on depth bit rate. The subjective evaluation showed that our proposed method achieves better quality in synthesized views than existing methods do. Subject terms: three-dimensional video; high efficiency video coding; view synthesis ; depth coding; edge-preserving upsampling. 1 Introduction The research and development in three-dimensional (3-D) video are capturing the attention of the research community, application developers, and the game industry. Many interesting applications of 3-D video—such as 3-D television (3DTV), free-viewpoint television, 3-D cinema, gesture recognition systems, and other consumer electronics products— have been developed. An attractive 3-D video representation is a multiview video plus depth (MVD) format, 1 which allows rendering numerous viewing angles from only two to three given input views. However, MVD results in a vast amount of data to be stored or transmitted, and efficient compression techniques for MVD are vital for achieving high 3-D visual experience with constrained bandwidth. In addition, the introduction of MVD format allows generating an arbitrary number of intermediate views with low-cost depth image–based rendering 2 techniques, but the quality depends on the accuracy of the depth maps. 3,4 Thus, in this article, we concentrate on the compression of depth information in an MVD format. A new video coding standard for high efficiency video coding (HEVC) 5 is now being finalized with a primary focus on efficient compression of monoscopic video. Preliminary results have already demonstrated that this new standard provides the same subjective quality at 50% of the bit rate compared to H.264/AVC High Profile. Recently, JCT-3DV has been formed for the development of new 3-D standards, including extensions of HEVC. Since depth maps generally have more spatial redundancy than natural …